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ABSTRACT 



The current emphasis on objectives and test item 
banks for constructing more effective tests is being augmented by 
increasingly sophisticated computer software. Items can be catalogued 
in numerous ways for retrieval. The items as well as instructional 
objectives can be stored and test forms can be selected and printed 
by the computer. It is also possible to select and retrieve 
objectives from which a curriculum can be created tailored to 
personal choice. This paper describes a very flexible item banking 
computer program and its use with a longitudinal, 

criterion-referenced testing program called Comprehensive Achievement 
Monitoring (CAM) , capable of both classroom management and curriculum 
evaluation. The CAM model includes item sampling techniques and 
several parallel test forms which would be particularly difficult to 
develop without the item banking computer programs. (Author) 
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Objective and Item Banking Computer Software 
and Its Use in Comprehensive Achievement Monitoring 

by 



Peter E . Schriber and William P. Gorth 
The University of Massachusetts 



Item banking and its logical predecessor objective banking are becoming 
increasingly important in educational measurement . Several projects, both 
in the U. S, and Britain, have investigated various forms of item banking. 

The motivation for these projects is usually the following; 

1) To make available to educators better test items for use in examina- 
tions in schools; 

2) To provide test items with known item characteristics so that test 
results will be more valid and reliable than those based on locally 
developed items and that test results from one setting can be com- 
pared with those from another setting; 

3) To make teachers more familiar, in general, with modern notions of 
test construction including the classification of test items by 
categories which they measure, e.g., behavioral objectives; 

4) To utilize test items, which are written by skilled authors, in 
many contexts without the added costs of writing new ones; and 

5) To provide a basis 'for better decision-making regarding the placement 
and instructional treatment of students; which will minimize losses 

in student T s time and effort. 

Objective and item banking require several operations including stocking 
of the bank, retrieving information from the bank, and using the retrieved 
information in a variety of testing situations. Each of these operations could 
be used to characterize existing or future objective and item banks. Stocking 
the bank consists of writing or compiling objectives and items which have been 
classified by content and characterized by item statistics. Retrieving from 
the bank consists of finding the objectives and items which are appropriate 
to the purpose for which they wilt be used. Using materials from the bank 
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consists of diagnostic testing, placement testing, criterion-referenced testing 
within a course, pre-testing for the different instructional treatments, or 
testing on a longitudinal basis using item sampling. 

Existing efforts in objective and item banking may be characterized by 
their purpose and operation. One of the most publicized efforts is the Instruc- 
tional Objective Exchange, IOX (Popham, 1970). IOX is an attempt to make 
available to teachers instructional objectives, with a grade level, content, 
and taxonomic.al classification of objectives, and sample test items. These 
materials are made available in the form of mimeographed booklets for specific 
subject areas and grade levels. The materials are not distributed in a form 
that can be used directly, i.e., the objectives are not printed in the form 
that could be transferred to a specific school situation, test items are not 
numerous enough or appropriately formated to constitute a test. IOX is not 
directed toward immediate implementation of objectives or testing programs 
but more as a guide toward development of locally based objectives and items. 

A second major effort would be the Computer-Based Test Development Center, 
COMBAT (Walter, 1970) . The major purpose of COMBAT is to make a large number 
of teacher written test questions available to classroom teachers for their 
classroom testing. The classification for test items is by key work and the 
item statistics are not measured. The storage and retrieval of the items is 
done by computer. They can be printed immediately in a form which can be used 
as a classroom test. The computer printing can be done on masters which dupli- 
cate more copies. The testing materials are designed for usual classroom 
testing . 

A more sophisticated bank, because extensive information about item char- 
acteristics are available, was produced at the National Foundation for Educa- 
tional Research in England and Wales. The work of the Foundation is described 
by Wood and Skurnik (1969). The item bank includes items which can be used 
by school-based examiners to determine the score of students in the certifi- 
cation of secondary education in England and Wales in mathematics. Extensive 
work went into the development of the item bank. Items were classified by 
task. They were pretested so that their item characteristics were known. The 
storage and retrieval of items is from a card file. 

Another effort in objective and item banking is the focus of this paper. 
The banking system has been developed by the Project for Comprehensive Achieve- 
ment Monitoring, CAM (Gorth, 1968). CAM has developed a model of evaluation 
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useful in curriculum evaluation and classroom management. The model consists 
of longitudinal testing,, using item sampling, of the specific behavioral objec- 
tives for a course. In order to support the testing activity, which uses a 
large number of test forms, and therefore, a large number of test items, com- 
puter programs were developed to streamline test development. 

All of the items in the CAM item bank have teen classified in at least 
three dimensions; their content, their taxonomical level, and the sequence 
in which they are taught in the typical school course. The relationships be- 
tween items and objectives are referenced and item analysis information at 
pretest, posttest, and retention time intervals is available. Both the objec- 
tives and the test items are stored on magnetic tape. The classification of 
items and their relation to the objective is also recorded in the computerized 
item bank. The objectives and items may be selected on a preliminary basis 
for perusal by individual teachers or for establishing a bank consisting of 
a subset of the total objectives and items. After final selection and allo- 
cation to test forms, the tests can be printed in an easily duplicatable format. 
The answer keys for the tests are both printed as well as punched into computer 
cards for quick analysis by other programs available from CAM (Gorth, Grayson, 
and Lindeman, 1969; Gorth, Grayson, and Stroud, 1969; Gorth, Grayson, Popejoy, 
and Stroud, 1969). 



The CAM Computer Programs 

Subsystem 1: Storage and Editing of Objectives and Items 

Input . The input into the computer program for storing objectives and 
items consists of data cards which contain up to 15 classifiers for the objec- 
tives and items and the text of the objectives and items in the format in which 
it will be later printed. The capability exists for modifying and adding to 
classifiers and texts of the objectives and items already stored in the banks. 

Output . The output from the computer program is a magnetic tape of the 
selected objectives and items and its printed listing. 

Subsystem 2: Preliminary Selection oC Objectives and Items 

Input . The second computer program selects objectives and their associ- 
ated test items by the criteria selected by the teacher. 

Output . The output consists of; 
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(1) A printout of the objective on one side of the printout page 
and the associated items listed on the other side. (See insert 
1 for an example.) The item is printed out in the format which 
would be used on a test and the correct answer is indicated by 
a leading asterisk. 

(2) A magnetic tape containing the objectives and items initially 
selected. This tape, containing the preliminary selection of 
objectives and items, is much shorter than the original tape 
and precludes having to search the master tape again for the 
same subset objectives and items before the tests are finally 
printed. The tape can be developed tc serve as a local bank 
tailored to the local needs and can be distributed for local 
use. An example is a school system with specified curricula 
for its science programs, so the tape \*Jould contain science ob- 
jectives and items selected as pertinent to the specified cur- 
ricula . 

Subsystem 3: Objective and Test Printing 

Input . The computer tape containing the subset of objectives and items 
constituting the selection by preliminary criteria by teachers is used as the 
major input for the final stage of the objective-item banking sequence. 

Output . The output consists of tests printed in a form which can be 
readily duplicated and administered. (See insert 2 for an example.) The tests 
may be printed on duplication masters or directly on multipart, computer output 
paper. Objectives associated with the tests are printed out in a form easy to 
duplicate and distribute to students. The objectives are coded with numbers 

selected by the teacher to fit the pattern and sequence of the curriculum as 

\ 

he organized it. 

Elaboration on the Use the Objective and Item Bank 
for the Construction of Tests 

The use of a computer program to compose and print actual test forms effec- 
tively reduces the chance of errors on the forms and saves hours of personnel 
time in the construction and reproduction of the forms. Errors introduced 
in preparing test masters and miskeying of answers are eliminated. 
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The program saves man-hours in its use of item sampling procedures where 
items must be randomly selected and randomly assigned to test forms. It also 
randomizes alternatives to multiple-choice items before they are printed out 
in the final version of the test and it further provides answer keys. In lon- 
gitudinal testing situations such as CAM where the use of multiple test forms 
is necessary, the labor saving is immense. 

Examples of Projects Using an 
Objective and Item Bank in Conjunction With a CAM System 
For a high school biology course in Duluth, Minnesota we have not only 
analyzed their test data but printed the test forms for the course by computer. 

Currently at the college level we have an item bank for a graduate educa- 
tional statistics course for which we print tests and analyze the data. 

A high school history course at Hopkins High School in Minnesota uses 
tests we have prepared by computer in a Title III project devoted to implemen- 
ting CAM on a school-wide basis. 

We are now in the process of building an objective and item bank in reading 
using material from the CAM project at Eallston Spa, New York. 

A fifth component of our current bank is a classroom evaluation course 
for teachers. For this course we print tests, analyze the data, and have the tea- 
chers in the course interpret their own results. 

Future Utility and Role of a 
Comprehensive Objective and Item Rank 
The use of an objective and item bank can go beyond the major desired 
results of providing a tailored curriculum for a teacher and efficient tests 
for student, course, and curriculum evaluations. These two results could be 
extended and enlarged through the organization of libraries of computer tapes 
which are subsets of the master library bank. Efficient means of tape creation, 
distribution, and continuous updating should be instituted so that the master 
bank could service hundreds of schools. Feedback from teachers who create 
new objectives and items and use existing bank materials in new situations 
could be a major component in bank modification. 

Non-access to computers could be bypassed by the distribution of the objec- 
tives and items on microfiche. 
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Such a bank should be an encouragement to teachers to buxld well- 
organized courses and curricula and yet ones which are flexible and diversified. 
Teachers need not create a curriculum in toto from bank materials but should 
be encouraged to pick and choose objectives and items to add to their own and to 
use the bank in determining an instructional sequence of objectives. The bank 
would then serve to lessen the effort needed to revise their curricula. The 
freedom to add or remove curricular segments in an easy manner would aid in 
lessening a teacher's dependence on one set of texts or the current collection 
of materials at hand. Further, it is hoped that motivation would arise to 
create learning situations for the previously refined and proven behavioral 
objectives which would be more effective for students than former practices 
based on a vaguer and less well organized curriculum. 

The common-core curriculum that could be formed if large numbers of schools 
used the banks resources could lead to statewide longitudinal testing programs 
which could be formulated with item and student sampling and serve as effec- 
tive substitutes for such state evaluation programs as the New York State 
regents tests. 
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More detailed information about the computer programs of the bank may 

be found in the following reference: > 

Gorth, W.F., and Grayson, A. Computer programs for objective and 
item banking. Educational and Psychological Measurement , 1971, 

31 , (In Press) . 
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of the following materials or sources would you 
CJnSI.iFr MOST RELIABLE to GIVE a true PICTURE *F the 

Battle of the mahne. September* i9ia. 

A. am ACCOilNl compiled from MATERIAL TAKEN FROM REPORTS OF 

opposing Colanders. 

U. AN account APPEARING IN A JOURNAL PRINTED IN A NEUTRAL 

country. 

l. an account which contrasted the spirit of the heroic boys 
- OF THE allies and the PCNoEROUS FCCLHaRDINeSS OF Thf 

enemy. 
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which of the following materials or sources would/ you 

- — consider most meltable to give a true picture of the 

Rhohlfms facing high school dropouts. 

a. a publication hy The u. s. department of labor on the 

1 limited J0 H OPPORTUNITIES AVAILABLE to DROPOUTS. 

d* A tv DOCUMENTARY On SchCCl DROPOUTS in WhIcH T£N DROPOUTS 

I are 'Interviewed* — - — — - — 

j c. an ARTICLE On the PRCrlEmS a DROPOUT MUST F A CE IN SCHOOL 
i and society magazine written by a famous sociologist. 



uuESTiom 3 

; WHICH - Or THE FOLLOWING Ma TeRTALS - OR- SOURCE S - WCULO YOU 

! CGhSIofR most reliable To GIVE a TRUE PICTURE of OWIGHT 

j. e i senhowsEr as a General. - 

-- ■ a . a u. s. army historians account of eisenhcwer*s military 
CAwEER. 

d. -A- MOVIE DtPlCTIMG-THE- LIFE OF GENERAL EISENHCwe«. 

c. a n account written hy his wife* mamie* about their years 
together. 



UUtSl ION A 

would -yCU-CLaSSIFy- the FOLLOWING item as A SOURCE- or a 

SlCCnDawY ACCOUNT for the EVENT IT DESCRIBES. 
a photograph of the burning of the airship hIndenburg. 






A. SECONDARY ACCOUNT. 

d. source. 



wuEST.ICN 5 . 

"COLD YOU CLASSIFY THe FOLLOWING ITEM AS A SOURCE 
stCCUD aR Y ACCOUNT FOR THE EVENT IT OESCRIBES. 

a mmpyrus roll used By the Egyptians. 



a. secondary account. 



ri 



OR a 



